Add RayService extra dependency example #4362
Conversation
Blaze-DSP
force-pushed
the
feat/extra-dependency
branch
from
de238a9 to
0654d28
Compare
| resources: | ||
| limits: | ||
| cpu: 2 | ||
| memory: 4Gi | ||
| requests: | ||
| cpu: 2 | ||
| memory: 4Gi |
There was a problem hiding this comment.
Done. Plus the example config has been updated, since there were issues with prv one
Signed-off-by: DPatel_7 <[email protected]>
| # Shared dependencies via args | ||
| # System packages and pip packages installed here are accessible | ||
| # to ALL applications in this RayService | ||
| args: | ||
| - | | ||
| sudo apt-get update && \ | ||
| sudo apt-get install -y --no-install-recommends curl && \ | ||
| sudo rm -rf /var/lib/apt/lists/* && \ | ||
| pip install httpx \ | ||
| # Make sure to add the \ at the end of the last line |
There was a problem hiding this comment.
Hi, thanks for the PR!
Using args for dependency installation might be risky.
Since args executes first, if the installation takes > 10 mins, the liveness probe will fail and restart the Pod.
Would postStart be a better approach here?
| # Shared dependencies via args | |
| # System packages and pip packages installed here are accessible | |
| # to ALL applications in this RayService | |
| args: | |
| - | | |
| sudo apt-get update && \ | |
| sudo apt-get install -y --no-install-recommends curl && \ | |
| sudo rm -rf /var/lib/apt/lists/* && \ | |
| pip install httpx \ | |
| # Make sure to add the \ at the end of the last line | |
| # Shared dependencies via lifecycle postStart hook | |
| # System packages and pip packages installed here are accessible | |
| # to ALL applications in this RayService | |
| lifecycle: | |
| postStart: | |
| exec: | |
| command: | |
| - /bin/bash | |
| - -c | |
| - | | |
| sudo apt-get update && \ | |
| sudo apt-get install -y --no-install-recommends curl && \ | |
| sudo rm -rf /var/lib/apt/lists/* && \ | |
| pip install httpx |
There was a problem hiding this comment.
It is better to do it before ray start, cz if it happens after ray and takes a long time, then the serving will fail incase, dependencies are not present when serve applications starts. I did try with bigger dependencies and it seemed to work for me.
ur thoughts on this?
There was a problem hiding this comment.
I don't think this is a good idea because
when installation > 10 min, kubelet will restart the pod, and we will repeat it again and again
Future-Outlier
left a comment
Future-Outlier
left a comment
There was a problem hiding this comment.
we should build a custom image instead, since this might cause other potential problem like here
cc @rueian
3 participants
Description
Add
ray-serve.extra-dependency.yamlexample demonstrating how to install custom dependencies in RayService containers. The example shows:argsruntime_env